Testing is an important activity, used widely by professional and end-user programmers alike in locating errors in their programs. In recognition of its importance and widespread use, there has been extensive research into effective testing in traditional programming languages in the imperative paradigm. However, there are few reports in the literature on testing in other paradigms, and no known reports on testing in spreadsheet languages other than that presented in U.S. patent application Ser. No. 09/438,084, entitled xe2x80x9cMETHODOLOGY FOR TESTING SPREADSHEETSxe2x80x9d, filed Nov. 10, 1999. The spreadsheet paradigm includes not only commercial spreadsheet languages, but also a number of research languages that extend the paradigm with explicitly visual features, such as support for gestural formula specification (Burnett98, Leopold97), graphical types (Burnett98, Wilde90), visual matrix manipulation (Wang96), high-quality visualizations of complex data (Chi98), and specifying GUIs (Myers91). In this specification, the term spreadsheet languages describes all such systems following the spreadsheet paradigm.
Despite the perceived simplicity of spreadsheet languages, and even though spreadsheet creators devote considerable effort to finding and correcting their errors (Nardi91), errors often remain. In fact, a recent survey of spreadsheet studies (Panko96) reports spreadsheet error rates ranging from 38% to 77% in controlled experiments, and from 10.7% to 90% in xe2x80x9cproductionxe2x80x9d spreadsheetsxe2x80x94those actually in use for day-to-day decision making. A possible factor in this problem is the unwarranted confidence creators of spreadsheets seem to have in the reliability of their spreadsheets (Wilcox97).
To help solve this problem, applicant""s Ser. No. 60/125,414 (Rothernel98) presents a testing methodology for spreadsheets at the cell level. The methodology provides feedback as to xe2x80x9ctestednessxe2x80x9d of cells in simple spreadsheets in a manner that is incremental, responsive, and entirely visual. However, scalability issues were not addressed in that previous work. Accordingly, a need remains to scale up the approach to support large grids of cells with shared or copied formulas.
One aspect of the invention is two methods for testing a spreadsheet region. In one method, referred to as the Straightforward approach, test elements are collected for each spreadsheet cell in the region. Each cell""s execution trace is tracked and, when a group of cells is validated, the subset of test elements that participated in the execution trace for each cell is marked as covered.
In the other method, referred to as the Region Representative approach, test elements are collected for a region representative, which represents all cells in the region. In the preferred embodiment, the test elements are du-associations. Each cell""s execution trace is tracked and, when a cell is validated, the subset of test elements in the region representative that participated in the execution trace for that cell is marked as covered.
Another aspect of the invention is a method for providing a user with feedback of the testedness of the spreadsheet cells in the region. The set of test elements for the region is maintained, and the subset of the set of test elements that have been covered is identified. Using the sizes of the set of test elements and the subset of the set, a testedness measure is calculated for each cell in the region, which is provided to the user.